Designing better deep networks and better reinforcement learning (RL) algorithms are both important for deep RL. This work focuses on the former. Previous methods build the network with several modules like CNN, LSTM and Attention. Recent methods combine the Transformer with these modules for better performance. However, it requires tedious optimization skills to train a network composed of mixed modules, making these methods inconvenient to be used in practice. In this paper, we propose to design \emph{pure Transformer-based networks} for deep RL, aiming at providing off-the-shelf backbones for both the online and offline settings. Specifically, the Transformer in Transformer (TIT) backbone is proposed, which cascades two Transformers in a very natural way: the inner one is used to process a single observation, while the outer one is responsible for processing the observation history; combining both is expected to extract spatial-temporal representations for good decision-making. Experiments show that TIT can achieve satisfactory performance in different settings, consistently.

Recently, there has been increasing interest in synthesizing data to improve downstream text-to-SQL tasks. In this paper, we first examined the existing synthesized datasets and discovered that state-of-the-art text-to-SQL algorithms did not further improve on popular benchmarks when trained with augmented synthetic data. We observed two shortcomings: illogical synthetic SQL queries from independent column sampling and arbitrary table joins. To address these issues, we propose a novel synthesis framework that incorporates key relationships from schema, imposes strong typing, and conducts schema-distance-weighted column sampling. We also adopt an intermediate representation (IR) for the SQL-to-text task to further improve the quality of the generated natural language questions. When existing powerful semantic parsers are pre-finetuned on our high-quality synthesized data, our experiments show that these models have significant accuracy boosts on popular benchmarks, including new state-of-the-art performance on Spider.

We represent the ResNeRF, a novel geometry-guided two-stage framework for indoor scene novel view synthesis. Be aware of that a good geometry would greatly boost the performance of novel view synthesis, and to avoid the geometry ambiguity issue, we propose to characterize the density distribution of the scene based on a base density estimated from scene geometry and a residual density parameterized by the geometry. In the first stage, we focus on geometry reconstruction based on SDF representation, which would lead to a good geometry surface of the scene and also a sharp density. In the second stage, the residual density is learned based on the SDF learned in the first stage for encoding more details about the appearance. In this way, our method can better learn the density distribution with the geometry prior for high-fidelity novel view synthesis while preserving the 3D structures. Experiments on large-scale indoor scenes with many less-observed and textureless areas show that with the good 3D surface, our method achieves state-of-the-art performance for novel view synthesis.

与传统的基于模型的故障检测和分类（FDC）方法相比，深神经网络（DNN）被证明对航空航天传感器FDC问题有效。但是，在训练中消耗的时间是DNN的过度，而FDC神经网络的解释性分析仍然令人难以置信。近年来，已经研究了一个称为基于图像缺陷的智能FDC的概念。这个概念主张将传感器测量数据堆叠到图像格式中，然后将传感器FDC问题转换为堆叠图像上的异常区域检测问题，这很可能很可能借用了机器视觉领域的最新进展。尽管在基于图像缺陷的智能FDC研究中声称有希望的结果，但由于堆叠图像的尺寸较低，使用了小的卷积核和浅DNN层，这阻碍了FDC性能。在本文中，我们首先提出了一种数据增强方法，该方法将堆叠的图像膨胀到更大的尺寸（与机器视觉领域中开发的VGG16网的通讯）。然后，通过直接对VGG16进行微调训练FDC神经网络。为了截断和压缩FDC净大小（因此其运行时间），我们在微调网上进行修剪。还采用了类激活映射（CAM）方法，以解释FDC NET的解释性分析以验证其内部操作。通过数据增强，VGG16的微调以及模型修剪，本文开发的FDC网络声称，在5个飞行条件下（运行时间26 ms），在4架飞机上，FDC精度为98.90％。 CAM结果还验证FDC Net W.R.T.它的内部操作。

在本文中，提出了一种新型的数据驱动方法，称为“增强图像缺陷”，用于飞机空气数据传感器（AD）的故障检测（FD）。典范飞机空气数据传感器的FD问题，开发了基于深神经网络（DNN）的边缘设备上的在线FD方案。首先，将飞机惯性参考单元测量作为等效输入，可扩展到不同的飞机/飞行案件。收集了与6种不同的飞机/飞行条件相关的数据，以在培训/测试数据库中提供多样性（可伸缩性）。然后提出了基于DNN的飞行条件预测的增强图像缺乏。原始数据被重塑为用于卷积操作的灰度图像，并分析并指出了增强的必要性。讨论了不同种类的增强方法，即翻转，重复，瓷砖及其组合，结果表明，在图像矩阵的两个轴上的所有重复操作都会导致DNN的最佳性能。基于GRAD-CAM研究了DNN的可解释性，这提供了更好的理解并进一步巩固DNN的鲁棒性。接下来，DNN型号，具有增强图像缺陷数据的VGG-16将针对移动硬件部署进行了优化。修剪DNN后，具有高精度（略微上升0.27％）的轻质模型（比原始VGG-16小98.79％），并获得了快速速度（时间延迟减少87.54％）。并实施了基于TPE的DNN的超参数优化，并确定了超参数的最佳组合（学习速率0.001，迭代时期600和批次尺寸100的最高精度为0.987）。最后，开发了基于Edge设备Jetson Nano的在线FD部署，并实现了飞机的实时监控。我们认为，这种方法是针对解决其他类似领域的FD问题的启发性。

Making sense of multiple modalities can yield a more comprehensive description of real-world phenomena. However, learning the co-representation of diverse modalities is still a long-standing endeavor in emerging machine learning applications and research. Previous generative approaches for multimodal input approximate a joint-modality posterior by uni-modality posteriors as product-of-experts (PoE) or mixture-of-experts (MoE). We argue that these approximations lead to a defective bound for the optimization process and loss of semantic connection among modalities. This paper presents a novel variational method on sets called the Set Multimodal VAE (SMVAE) for learning a multimodal latent space while handling the missing modality problem. By modeling the joint-modality posterior distribution directly, the proposed SMVAE learns to exchange information between multiple modalities and compensate for the drawbacks caused by factorization. In public datasets of various domains, the experimental results demonstrate that the proposed method is applicable to order-agnostic cross-modal generation while achieving outstanding performance compared to the state-of-the-art multimodal methods. The source code for our method is available online https://anonymous.4open.science/r/SMVAE-9B3C/.

Spatio-temporal machine learning is critically needed for a variety of societal applications, such as agricultural monitoring, hydrological forecast, and traffic management. These applications greatly rely on regional features that characterize spatial and temporal differences. However, spatio-temporal data are often complex and pose several unique challenges for machine learning models: 1) multiple models are needed to handle region-based data patterns that have significant spatial heterogeneity across different locations; 2) local models trained on region-specific data have limited ability to adapt to other regions that have large diversity and abnormality; 3) spatial and temporal variations entangle data complexity that requires more robust and adaptive models; 4) limited spatial-temporal data in real scenarios (e.g., crop yield data is collected only once a year) makes the problems intrinsically challenging. To bridge these gaps, we propose task-adaptive formulations and a model-agnostic meta-learning framework that ensembles regionally heterogeneous data into location-sensitive meta tasks. We conduct task adaptation following an easy-to-hard task hierarchy in which different meta models are adapted to tasks of different difficulty levels. One major advantage of our proposed method is that it improves the model adaptation to a large number of heterogeneous tasks. It also enhances the model generalization by automatically adapting the meta model of the corresponding difficulty level to any new tasks. We demonstrate the superiority of our proposed framework over a diverse set of baselines and state-of-the-art meta-learning frameworks. Our extensive experiments on real crop yield data show the effectiveness of the proposed method in handling spatial-related heterogeneous tasks in real societal applications.

Fully convolutional detectors discard the one-to-many assignment and adopt a one-to-one assigning strategy to achieve end-to-end detection but suffer from the slow convergence issue. In this paper, we revisit these two assignment methods and find that bringing one-to-many assignment back to end-to-end fully convolutional detectors helps with model convergence. Based on this observation, we propose {\em \textbf{D}ual \textbf{A}ssignment} for end-to-end fully convolutional de\textbf{TE}ction (DATE). Our method constructs two branches with one-to-many and one-to-one assignment during training and speeds up the convergence of the one-to-one assignment branch by providing more supervision signals. DATE only uses the branch with the one-to-one matching strategy for model inference, which doesn't bring inference overhead. Experimental results show that Dual Assignment gives nontrivial improvements and speeds up model convergence upon OneNet and DeFCN. Code: https://github.com/YiqunChen1999/date.

对比性语言图像预训练（剪辑）通过随时可用的自然语言监督学习丰富的表示。它可以改善下游视觉任务的一般性能，包括但不限于零射击，长尾巴，细分，检索，标题和视频。但是，据我们所知，尚未研究剪辑的视觉解释性。为了提供其预测的视觉解释，我们提出了图像文本相似性图（ITSM）。基于它，我们出人意料地发现，剪辑比前景更喜欢背景区域，并且对人类理解提出了错误的可视化。在实验上，我们发现魔鬼在汇总部分，其中不适当的合并方法导致一种称为语义转移的现象。为了纠正和提高可视化结果，我们提出了蒙版的最大池，并使用自我监督图像编码器的注意力图。同时，解释性任务和识别任务需要不同的表示。为了解决这个问题，我们提出了双重预测，以满足这一要求。我们将上述方法整合为可解释的对比度图像预训练（ICLIP）。实验表明ICLIP极大地提高了可解释性。例如，在VOC 2012数据集中，非平凡的改进分别为$ 32.85 \％$和$ 49.10 \％$。

知识密集型语言任务（苏格兰信）通常需要大量信息来提供正确的答案。解决此问题的一种流行范式是将搜索系统与机器读取器相结合，前者检索支持证据，后者检查它们以产生答案。最近，读者组成部分在大规模预培养的生成模型的帮助下见证了重大进展。同时，搜索组件中的大多数现有解决方案都依赖于传统的``索引 - retrieve-then-Rank''管道，该管道遭受了巨大的内存足迹和端到端优化的困难。受到最新构建基于模型的IR模型的努力的启发，我们建议用新颖的单步生成模型替换传统的多步搜索管道，该模型可以极大地简化搜索过程并以端到端的方式进行优化。我们表明，可以通过一组经过适当设计的预训练任务来学习强大的生成检索模型，并被采用以通过进一步的微调来改善各种下游苏格兰短裙任务。我们将预训练的生成检索模型命名为Copusbrain，因为有关该语料库的所有信息均以其参数进行编码，而无需构造其他索引。经验结果表明，在苏格兰语基准上的检索任务并建立了新的最新性能，Copusbrain可以极大地超过强大的基准。我们还表明，在零农源和低资源设置下，科体班运行良好。